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DETAILED ACTION 

1 . The amendment filed 12/27/06 has been received and considered. Claims 1-21 
are presented for examination. 



Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
Invention was made to a person having ordinary skill In the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner.in which the invention was made. 

3. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 . 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

4. Claims 1-9, 11-14, and 17-21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Patrick Pirim, (Pirim hereinafter), U.S. Patent 7,136,842 taken in view 
of Stauffer and Grimson, (Stauffer hereinafter). Learning Patterns of Activity Using Real 
Time Tracking, and further in view of Lee, Romano, and Stein, (Lee hereinafter). 
Monitoring activities from multiple video streams: Establishing a common coordinate 
frame. 
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5. As to claim 1 , Pirim discloses a method for modeling movement of users in an 
environment (see col. 1, lines 60-65), the environment including sensors connected in a 
network (see "sensors" as "CMOS imaging devices" in col. 21, lines 51-56 and Fig. 35), 
comprising: detecting events due to movement (see col. 22, lines 42-44) of the users at 
the sensors (see "sensors" as "CMOS imaging devices" in col. 21, lines 51-56 and Figs. 
34 and 35); labeling each event according to a particular sensor that detected the event 
and time of the event (see "labeling" as "the parameter DPyt is ascribed the value", 
"each event" as "DPyt", and "time of the event" as "the values Alyt depend on a time 
constant" in col. 22, line 42 to col. 23, line 2); summing the events for each sensor into a 
corresponding histogram time interval bin associated with the sensor (see col. 2, lines 
18-27). While Pirim discloses modeling movement of users in an environment, Pirim 
fails to disclose generating a plurality of co-occurrence matrices from the histograms. 
Stauffer discloses generating a plurality of co-occurrence matrices from the histograms 
according to Cg, 6 = t=o I ^ Hi,t Hj,t+6 (see page 753, col. 2, Section No. 7.3, 1st and 3rd 
paragraphs and formula (13)),where i and j represent each possible pair of sensors (see 
page 753, col, 2, lines 1-2 and "each possible pair" in page 753, col. 2, 2nd paragraph), 
T is a total time for the detecting (see "Ctotai and Z" in page 753, col. 2, lines 1-20), t 
represents a particular time, and H represents the histogram time interval bins (see 
"PcO" in page 753, col, 1, next to last paragraph, lines 3-4). Therefore, it would have 
been obvious to one of ordinary skill in this art at the tinie of invention by applicant to 
utilize the co-occurrence matrices of Stauffer in the method of Pirim because Stauffer 
utilizes a probabilistic method for background subtraction, which involves modeling each 
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pixel as a separate mixture model (see page 757, col. 1. 3^^ paragraph from the bottom, 
lines 1-3), and as a result, Stauffer reports successful use in tracking people in indoor 
environments, people and cars in outdoor environments, fish in a tank, ants on a floor, 
and remote control vehicles in a lab setting; using different cameras, using different 
lighting, and tracking different objects, while achieving real-time performance over 
extended periods of time without human intervention (see page 757, col. 1, last 
paragraph). 

6. While the Pirim-Stauffer method discloses almost all of the instant invention, the 
Pirim-Stauffer method lacks time calibration via time-offsets. Lee discloses time-offsets, 
i.e. 6 in the co-occurrence matrices, (see page 761, col. 2, Section No. 4,4, 1st 
paragraph). 

7. Pirim, Stauffer, and Lee are analogous art because they are both related to 
modeling activity using video streams. 

8. Therefore, it would have been obvious to one of ordinary skill in this art at the 
time of invention by applicant to utilize the time-offsets of Lee in the Pirim-Stauffer 
method because Lee models the activity in a large site using video streams from 
multiple cameras (see page 758, col. 1. Introduction, lines 1-3), and as a result, reports 
the following improvements over his prior art: an unsupervised method for coordinating 
the activities detected in a distributed, uncalibrated, set of cameras into a single global 
coordinate frame, which uses tracking data from moving objects to solve the 
correspondence problem between cameras, and which allows multiple views to be 
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coordinated to a single view or to a global, ground plane view (see page 767, col. 1 , 
Conclusions, 2nd paragraph, lines 1-7). 

9. As to claim 2, Stauffer discloses a method in which the environment is inside a 
building (see "indoor sites" in page 747, col. 2, lines 1-2). 

10. As to claim 3, Stauffer discloses a method in which the environment is an open 
space (see "outdoor sites" in page 747, col. 2, line 3). 

11. As to claims 4-6, Stauffer discloses a method in which the network is 
wired/wireless and includes wired and wireless sensors (see page 747, col. 2, lines 2- 
3). 

12. Claims 4-6 have been given a broad reasonable interpretation by the Examiner. 
The Examiner notes that the steps disclosed in (page 747, col. 2, lines 2-3) are 
functionally equivalent to the results produced by the steps expressly claimed in 
Applicant's dependent claims 4-6. Therefore, the "product" that is produced by 
performing the steps disclosed in dependent claims 4-6 is the functional equivalent of 
the "product" that is produced in (page 747, col. 2, lines 2-3). Although the "steps" by 
which the end result is different, the final result for the "steps" is identical. 

13. As to claim 7, Stauffer discloses a method in which the sensors have a resolution 
of one bit to detect the events (see "binary" in page 752, col. 1 , Section No. 6, lines 4- 
6). 

14. As to claim 8, Stauffer discloses a method in which a sensing modality is 
selected from the group consisting of infra-red, thermal, ultrasonic, light, radar, sonar. 
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microwave, pressure, on-off switch, and combinations ttiereof (see "lighting" in page 
757, col. 1 , next to last paragraph, lines 1-2). 

15. As to claim 9, Stauffer discloses a method in which the sensors are globally 
distributed in the environhient in an ad-hoc arbitrary pattern (see page 747, col. 1, last 
lines 3). 

16. As to claim 1 1 , Stauffer discloses a method comprising: determining, for each . 
possible pair of sensors, a distance between the pair of sensors (see "Self-calibration" 
in page 747, col. 2, lines 11-12); and mapping the distances to geometry of the sensors 
(see page 747, col. 2, lines 13-14). 

17. As to claim 12, Lee discloses a method in which the summed events include 
peaks, and each peak reflects a relative time-offset between a particular pair of sensors 
(see page 761, col. 2. Section No. 4.4, 1st paragraph, last 5 lines and Fig. 2). 

1 8. As to claim 1 3, Stauffer discloses a method in which the detemriining further 
comprises: applying a consistent average velocity of the movement of the users over 
time to the relative time-offset between the pair of sensors (see "speed" in page 752, 
col. 1 , Section No. 6, lines 2-4). 

19. As to claim 14, Stauffer discloses a method in which the mapping uses multi- 
dimensional scaling (see "speed" in page 752, col. 1 , Section No. 6, lines 2-4). As per 
"By applying an estimated velocity of the users, the geometry can be scaled to real- 
world dimensions" in (application description page 11, paragraph [055], last 2 lines), 
Examiner interprets multi-dimensional scaling as estimated velocity. 
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20. As to claim 17, Stauffer discloses a method in which a particular sensor detects 
events associated with predicting a future activity, and further comprising: determining, 
from the co-occurrence matrices, distributions of probabilities that the terminating event 
occurs based on the events detected by all of the other sensors other than the particular 
sensor (see page 754. col. 1. lines 9-18); and summing the distributions of probabilities 
to predict the probability that the future activity will occur prior to detecting the 
terminating event while detecting the events (see page 754, col. 1, last paragraph). 

21 . As to claim 18, Stauffer discloses a method in which there is one distribution for 
each other sensor (see "Once the parameters for the pmfs have been determined, any 
exclusive set of them can be used as classifiers" in page 754, col. 1, last paragraph, . 
lines 1-2). 

22. Claim 18 has been given a broad reasonable interpretation by the Examiner. The 
Examiner notes that the step disclosed in (page 754, col. 1, last paragraph, lines 1-2) is 
functionally equivalent to the results produced by the step expressly claimed in 
Applicant's dependent claim 18. Therefore, the "product" that is produced by performing 
the step disclosed in dependent claim 18 is the functional equivalent of the "product" 
that is produced in (page 754, col. 1, last paragraph, lines 1-2). Although the "step" by 
which the end result is different, the final result for the "step" is identical. 

23. As to claims 19-21, these claims recite a system for performing the Stauffer-Lee 
method of claims 1,11, and 17. Stauffer discloses a system (see page 747, col. 1, lines 
1-3) for performing a method that teaches claims 1,11, and 17. Therefore, claims 19- 
21 are rejected for the same reasons given above. 
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24. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pirim 
taken in view of Stauffer further in view of Lee as applied to claim 14 above, and further 
in view of Yaron Caspi and Michal Irani, (Caspi hereinafter), Alignment of non- 
overlapping sequences 

25. As to claim 10, while the Pirim-Pirim-Stauffer-Lee method teaches almost all of 
the instant invention as applied to claim 14 above, the Pirim-Stauffer-Lee method lacks 
non-overlapping sensor ranges. 

26. Caspi discloses a method in which ranges of some sensors are non-overlapping 
(see page 5, col. 2, last paragraph, lines 1-6). 

27. Pirim, Stauffer, Lee, and Caspi are analogous art because they are related to 
modeling activity using video streams. 

28. Therefore, it would have been obvious to one of ordinary skill in this art at the 
time of invention by applicant to utilize the non-overlapping sensor ranges of Caspi in 
the Pirim-Stauffer-Lee method because Caspi shows how two image sequences that 
have no spatial overlap between their fields of view can be aligned both in time and in 
space (see page 1, col. 1, lines 1-3), and as a result, Caspi reports the following 
improvement over his prior art: sequence alignment can be used to generate wide- 
screen movies from two (or more) narrow field-of-view movies (such as in I MAX movies) 
(see page 6, col. 2, last paragraph, lines 1-3). 
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29. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Pirim taken in view of Stauffer further in view of Lee as applied to claim 14 above, 
and further in view of Mark Steyvers, (Steyvers hereinafter). Multidimensional Scaling 
(see IDS dated 10/10/03). 

30. As to claim 1 5. while the Pirim-Stauffer-Lee method teaches almost all of the 
instant invention as applied to claim 14 above, the Pirim-Stauffer-Lee method lacks 
using a stress measure. 

31. Steyvers discloses a method comprising: evaluating a likelihood that a particular 
geometry conforms to the estimated distances using a stress measure O = I [dy- 
f(5ij)],where f(5ij) is a non-metric, monotone transformation (see page 2, col. 1, lines 1- 

3). 

32. Claim 15 has been given a broad reasonable interpretation by the Examiner. The 
Examiner notes that it would have been obvious to one of ordinary skill in this art at the 
time of invention by applicant to utilize manipulation of mathematical concepts. 
Examiner notes that f(5ij) is defined as a non-metric, monotone transformation. Steyvers 
discloses a stress measure formulation S with f(6y), a non-metric monotone 
transformation, (see "dY in page 2, col. 1, lines 7-11). Additionally. Steyvers discloses 
that since 1964 (see page 1, col. 2, last 3 lines) "Several iterative minimization 
algorithms exist to move the object points in a multidimensional space in order to 
minimize stress" (see page 2, col. 1, lines 11-15). 

33. Pirim, Stauffer, Lee, and Steyvers are analogous art because they are related to 
modeling activity using video streams. 
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34. Therefore, it would have been obvious to one of ordinary skill in this art at the 
time of invention by applicant to utilize the stress measure of Steyvers in the Pirim- 
Stauffer-Lee method because Steyvers utilizes Multidimensional Scaling (MDS), which 
describes techniques for proximity data analysis on a set of stimuli to reveal the hidden 
structure underlying the data (see page 1, col. 1. lines 1-4), and as a result, Steyvers 
reports the following applications of MDS: multidimensional representations resulting 
from MDS are also often useful as the representational basis for various 
mathematical models of categorization and/or identification (see page 1, col. 2, 2nd 
paragraph, lines 6-11). 

35. As to claim 16, Steyvers discloses a method in which the stress measure rank- 
orders the distances between the pairs of sensors (see page 3, col. 1, lines 4-9). 

Response to Arguments 

36. Applicant's arguments filed 12/27/06 have been fully considered but they are not 
persuasive. 

37. Regarding the specification objections, the amendment corrected all deficiencies 
and the objections are withdrawn. 

38. Regarding the claim objections, the amendment corrected all deficiencies and 
the objections are withdrawn. 

39. Regarding the rejection under 103. Applicant's arguments with respect to claims 
have been considered but are moot in view of the new grpund(s) of rejection. In the 
instant rejection, Examiner has elaborated prior art disclosures of amended claims. 
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40. Applicant argues, (see page 13, last 4 paragraph), that "Applicant cannot find the 
claimed stress measure formulation cj) = z [dij-f(6,j)], in Steyvers". Examiner notes that 
f(6ij) is defined as a non-metric, monotone transformation. Steyvers discloses a stress 
measure formulation S with f(5jj), a non-metric monotone transformation, (see "d*ij" in 
page 2, col. 1. lines 7-11). Additionally, Steyvers discloses that since 1964 (see page 1, 
col. 2, last 3 lines) "Several iterative minimization algorithms exist to move the object 
points in a multidimensional space in order to minimize stress" (see page 2, col. 1, lines 
11-15). Claim 15 has been given a broad reasonable interpretation by the Examiner. 
The Examiner notes that it would have been obvious to one of ordinary skill in this art at 
the time of invention by applicant to utilize manipulation of mathematical conc.epts. 

Conclusion, 

41 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

42. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

43. Christopher R. Wren and Srinivasa G. Rao, "Self-Configuring, Lightweight 
Sensor Networks For Ubiquitous Computing". (See PTO-892 Notice of Reference Cited 
dated 9/27/06) teaches all contents of this application (see page 2. line 1). Examiner 
previously enquired about the exclusion of Mr. Srinivasa G. Rao and Applicant was 
non-responsive. 

44. Examiner would like to point out that any reference to specific figures, columns 
and lines should not be considered limiting in any way, the entire reference is 
considered to provide disclosure relating to the claimed invention. 

45. Any inquiry concerning this communication or eariier communications from the 
examiner should be directed to Juan C. Ochoa whose telephone number is (571) 272- 
2625. The examiner can normally be reached on 7:30AM - 4:00 PM. 

46. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Paul Rodriguez can be reached on (571) 272-3753. The fax phone number 
for the organization vyhere this application or proceeding is assigned is 571-273-8300. 

47. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
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For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 






